Performing security testing on a computer system might involve exercising parts of the functionality of the computer system and evaluating whether an instance of security vulnerability exists. For example, if a computer system is supposed to be accessible only to authorized persons and is supposed to block unauthorized persons, a simple test might be for a tester to access the computer system and at a login screen that asks for a user name and a password, type in a known invalid name, such as “nobody” and a known invalid password such as leaving the password field blank and then submitting that as a login. If the computer system responds by allowing the tester past the login screen as if the tester were an authorized person, that indicates that the computer system has a security vulnerability. If the computer system responds by displaying a message such as “Unrecognized user name or password” and remains at the login screen, that may indicate that the computer system might not have that particular vulnerability.
Some security testing might be done manually, wherein a human tester tries to execute a few operations known to expose common vulnerabilities and then explores different attack vectors to identify more subtle vulnerabilities. This can involve considerable amount of effort and can be more difficult if not all aspects of the system under test are accessible.
For a mobile application (a program designed to run on a mobile device or other reduced-functionality device, such as a smartphone, tablet, smartwatch, etc., or designed to run in a simplified environment rather than a general purpose computing system; often referred to as an “app”), its vulnerabilities might be discovered through manual analysis of the source code of the app or of the executable binary code of the app. Both approaches have advantages, the latter case being the most desirable for detecting vulnerabilities in the product that the user sees. However, analysis of the binary code presents many difficulties for the analyst, in part because the binary code lacks information for understanding the semantics of the app's logic and the hooks needed to trigger functionality for automation of vulnerability scanning.
One approach for automated vulnerability scanning is static testing, wherein a testing system or program reads the binary code and looks for patterns known to be associated with vulnerabilities. Another approach is dynamic analysis, where the app is executed on a suitable platform and an analyst examines the external response of the app to inputs (e.g., the app's outputs, the network traffic, file input/output, etc.) and/or internal behavior (e.g., code injection, uncleared sensitive data, etc.) and deduces vulnerabilities from that behavior. Hybrid approaches are also possible, such as static testing with limited emulation of subcomponents. In any case, an automated system or analyst is limited to derivation of what the binary code is expected to do.
Semi-automated or fully-automated tools to perform static/dynamic testing might be driven by an analyst's intuition, but what often happens is that the tools are either too specific in their vulnerability signature or output too many false positives. In either case, the analyst potentially ends up with more work than initially expected.